360 x 360 Mosaics

Current mosaicing methods use narrow field of view cameras to acquire image data. This poses problems when computing a complete spherical mosaic. First, a large number of images are needed to capture a sphere. Second, errors in mosaicing make it difficult to complete the spherical mosaic without seams. Third, with a hand-held camera it is hard for the user to ensure complete coverage of the sphere. This paper presents two approaches to spherical mosaicing. The first is to rotate a 360 degree camera about a single axis to capture a sequence of 360 degree strips. The unknown rotations between the strips are estimated and the strips are blended together to obtain a spherical mosaic. The second approach seeks to significantly enhance the resolution of the computed mosaic by capturing 360 degree slices rather than strips. A variety of slice cameras are proposed that map a thin 360 degree sheet of rays onto a large image area. This results in the capture of high resolution slices despite the use of a low resolution video camera. A slice camera is rotated using a motorized turntable to obtain regular as well as stereoscopic spherical mosaics. 1 Spherical Mosaics A mosaic is constructed by stitching1 together multiple images, where the individual images correspond to different views of the scene captured from approximately the same viewpoint. Several methods for image mosaicing have been proposed (for examples, see [Burt and Adelson, 1983], [Mann and Picard, 1994], [Zheng and Tsuji, 1992], [Chen, 1995], [Irani et al., 1995], [Kang and Szeliski, 1996] [Peleg and Herman, 1997], [Rousso et al., 1997], [Szeliski, 1996], [Sawhney et al., 1995], [Krishnan and Ahuja, 1996], [McMillan and Bishop, 1995], [Szeliski and Shum, 1997]). These techniques use a conventional imaging lens to capture the image sequence. Since such lenses have limited fields of view, the computation of a complete spherical mosaic requires the capture and processing of a large number of images. In addition, errors in the image projection model and errors in the estimation of motion between images makes it difficult to complete the sphere This work was supported in parts by DARPA’s Image Understanding Program and an ONR/DARPA MURI grant under ONR contract No. N00014-97-1-0553. 1In our definition of mosaicing, we will include both image based as well as slit (a slice through the image) based techniques. In the case of slits, the slices are not really stitched but rather concatenated together to form the mosaic. without undesirable seams in the final mosaic. Further, in the case of a hand-held camera, it is hard for the user to ensure that the complete sphere has been scanned during the capture process. An alternative approach is to use a wide-angle imaging system such as a fish-eye lens (see [Kuban et al., 1994], [Xiong and Turkowski, 1997]) or a catadioptric imaging system (see [Nayar, 1997], [Yagi, 1999] for surveys). In both cases, a hemispherical field of view can be captured within a single image. Hence, a small number of such images can be stitched together to obtain a spherical mosaic2. However, this approach typically results in inadequate resolution due to the inherent trade-off between field of view and image resolution; as the field of view increases, the resolution decreases, causing the computed spherical image to be of lower quality than in the case of a conventional imaging system. This paper presents two efficient approaches for capturing high resolution spherical mosaics. In the first approach, a wide-angle imaging system is used to capture a sequence of 360 degree strips on the sphere by a single rotation of the capture device. For this, we suggest the use of a catadioptric imaging system since such a system typically produces higher resolution in the periphery of the hemispherical field of view than a fish-eye lens. The unknown rotations between the strips are estimated and used to blend the multiple strips into a single spherical mosaic. Our second approach seeks to further enhance the resolution of the computed mosaic. This is done by designing new catadioptric sensors that capture a single 360 degree slice of the scene3. Mirror shapes are derived that enable the projection of a thin slice onto a large image area. This results in the capture of high resolution slices despite the use of a low resolution (640x480 pixel) image detector. Such a slice camera is rotated on a turntable and the captured slices are concatenated to obtain a high resolution spherical mosaic. Though a large number of images (slices) are needed to obtain a high resolution mosaic, the processing of each image is minimal and is easily done in real time. Recently, several investigators have explored the capture of stereoscopic panoramas. Ishiguro et al. [Ishiguro et al., 1992] were the first to use stereo panoramas for computing structure. Then, Huang and Hung [Huang and 2See [IPIX, 1999] for results on the stitching of two fish-eye images to obtain a spherical mosaic. 3In [PanoScan, 1999], a 180 degree slice is captured by using a fisheye lens and a high resolution line detector. 1063-6919/00 $10.0